Calibration Modeling Methodology to Optimize 
Performance for Low Range Applications 

Raymond A. McCollum * 

Booz Allen Hamilton, Norfolk, VA, 23502 

Sean A. Commo t and Peter A. Parker * 

National Aeronautics and Space Administration, Hampton, VA, 23681 


Calibration is a vital process in characterizing the performance of an instrument in an ap- 
plication environment and seeks to obtain acceptable accuracy over the entire design range. 
Often, project requirements specify a maximum total measurement uncertainty, expressed 
as a percent of full-scale. However in some applications, we seek to obtain enhanced perfor- 
mance at the low range, therefore expressing the accuracy as a percent of reading should be 
considered as a modeling strategy. For example, it is common to desire to use a force bal- 
ance in multiple facilities or regimes, often well below its designed full-scale capacity. This 
paper presents a general statistical methodology for optimizing calibration mathematical 
models based on a percent of reading accuracy requirement, which has broad application 
in all types of transducer applications where low range performance is required. A case 
study illustrates the proposed methodology for the Mars Entry Atmospheric Data System 
that employs seven strain-gage based pressure transducers mounted on the heatshield of 
the Mars Science Laboratory mission. 
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E Estimated covariance matrix 

a Mean squared error of model 

Subscript 

i Index Number 


I. Introduction 

This paper demonstrates the use of linear model theory to characterize the performance of an instrument. 
A wealth of literature is available on how to manipulate and control linear model estimates and corresponding 
prediction intervals. Typically, linear model theory is associated with minimizing the errors across the entire 
design space. The problem addressed here is how to adapt linear model theory when a particular region 
of the design space is more important. In particular, this paper focuses on low-range applications of linear 
model theory while providing a general methodology that can be applied to other calibration exercises. 

The example presented in this paper addresses the calibration of seven strain- gage based pressure trans- 
ducers that will used as a part of the Mars Entry Atmospheric Data System (MEADS). MEADS will measure 
the pressure distribution across the heatshield during the Mars Science Laboratory’s entry into the Mars at- 
mosphere. Because the predicted measurements during the trajectory are dominated by low- range pressures, 
calibration requirements for the project were defined in terms of reading error. 

II. Methodology 

Standard linear model theory minimizes the difference between an actual, measured response and a 
predicted response . 1 For the MEADS project, this difference is between the measured output signal and 
the output signal estimated by the calibration model. Generalized least squares, which is used to develop 
the baseline calibration model for the transducers, weights the errors across the design space equally. These 
errors are often re-expressed in terms of the full-scale error of the instrument. By definition, the full-scale 
error is 


Full-Scale Error = 


Actual Response — Predicted Response 
Full-Scale (Max) Response 


However, the requirements specified by the MEADS project are expressed in terms of reading error and not 
full-scale error. 


A. Reading Error 

There are occasions when the error in a particular region of the design space is more important than another 
region. For example, reading error is associated with improved prediction performance at low-end regions. 
Reading error is defined as 

_ _ . ^ Actual Response — Predicted Response 

Reading Error = — 

Actual Response 

It can easily be seen that the error between the actual and predicted response should be very small as 
the actual response approached zero. Therefore, for a constant reading error, the full-scale error must be 
significantly smaller for low-end responses. 

B. Design Space 

In traditional response surface methodology, the desired model dictates the experimental design. For ade- 
quate characterization of the MEADS pressure transducers, the following second order model was proposed, 
which is based on a second-order Taylor series approximation 


v — 00 + $ 1 P + $ 2 T sens + /?3 TsSE + $11 P 2 + $ 22 Tsens + $33^SSE + $ 12 P x Tsens + 

$13 P x TsSE + $23? sens x TsSE + € + S 


(1) 
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where e and 5 are the subplot and whole plot error terms, respectively. The presence of two error terms 
signals a restriction in the randomization sequence of the experimental design. 2 These restrictions are a 
result of increasing experimental efficiency by limiting the number of costly factor changes. Because of the 
restrictions on randomization, different experimental designs can be used for the two groups of factors as 
long as the designs support the desired model. For the temperature design space, a popular, two-factor 
spherical central composite design is used. The design consists of nine unique points, which support a full 
second order model. The nine unique temperature combinations, shown in Figure 1, are set in a random 
order. 



Figure 1. Temperature Design Space 


In addition to the nine unique points, there are single replicates of the four factorial points and at the 
extremes along the sensor temperature axis at (-130,0) and (-50,0). Five replicates at the center of the 
design space at (-90,0) are also included. This replication strategy provides a distributed 11 degrees of 
freedom to estimate the experimental error in temperature. 

Once a temperature combination is set, pressure settings are completely randomized. Only three unique 
levels are required to support a second-order model in pressure. However, from a practical point of view, 
pressure transducer calibration typically includes more than three points. Therefore, to enhance the pre- 
diction power of the model, two more unique pressure levels are included in the design. Dispersing the 
unique levels conveniently throughout the design space, the eight calibration points are 0.01, 0.12, 0.50, 079, 
1.00, 2.50, 4.00, and 5.00 psia. Replication at 0.01, 2.50, and 5.00 psia are included to estimate constant 
temperature repeatability of the transducers. The pressure design space is shown in Figure 2. 

C. Calibration Model 

The objective of the calibration is to develop a mathematical relationship between the output voltage from 
the transducer from the known applied pressure and temperature. However, when the transducer is used 
on flight data, the calibration equation is inverted so that pressure can be estimated from the measured 
voltage and temperature. Hence, the experiment used pressure and temperature as the predictor variables 
and voltage as the response variable. Once an equation was formed that predicted voltage, the equation 
would be inverted to gain an estimate of pressure. 

Voltage error divided by pressure gave an error ratio proportional to reading error. The objective was to 
use the linear models to minimize the ratio. Minimizing the ratio would also minimize the inverse solution. 
If the equation could predict voltage accurately, the inverse equation would predict pressure accurately. 
Analysis revealed that the pressure error could be approximated by dividing the voltage error by two. 
Mathematical and physical representation of the model terms are defined as 3 

Intercept term and it represents the voltage offset from zero. 
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Figure 2. Pressure Design Space 


Linear effect of pressure on the response and it represents the primary sensitivity of the transducer. 

/?2 Linear effect of the transducer temperature on the response and it represents the zero intercept ad- 
justment as a function the transducer temperature. 

fin Second-order effect of pressure (non-linearity) on the response. 

/?13 Primary sensitivity (pressure) adjustment as a function of SSE temperature on the response. 

/?22 Second-order effect of transducer temperature on the response. 

e Subplot error and it represents the variability of the transducer at constant temperature. 

5 Whole plot error and it represents the variability of the transducer due to changing temperature. 

Analysis shows that the whole plot error is significantly larger than the subplot error. This means every 
time the temperatures is changed, a new random draw is chosen from a normal distribution with a variance 
an order of magnitude above the subplot variance. Changing the pressure also cased a different random 
error to occur but the magnitude of that error is much smaller on average than the effect of changing the 
temperature. 


III. In-Flight Zero 

The error term for each set of predictor variables in the design space can be thought of as changing the 
intercept for the equation. Hence, the error term moves the result up or down for each pressure value. Each 
time the temperatures are changed, the result also increases or decreases with a higher magnitude. 

Each time the temperature values are changed there is a new random draw from the whole plot error 
distribution and that value is added to or subtracted from the voltage. One approach to compensate for 
this temperature variance was to use knowledge of the spacecraft before it enters the Martian atmosphere. 
Before entry, the spacecraft will be in the vacuum of space and hence have a pressure value of zero. In this 
instance, all the model variables are known including the voltage, both temperatures, and the pressure which 
is zero. Hence, it would be possible to use the predicted equation and determine an estimate of the intercept 
adjustment to the system prior to entry. 

In theory, this should remove most of the larger whole plot error and leave only the subplot error which is 
a result of pressure changes. Since engineering data suggests there will be only small changes in temperature 
during entry, the whole plot error should remain relatively constant. The small change in temperature will 
produce a negligible effect on the whole plot error and it should remain relatively constant. 
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IV. Derivation 


Linear model theory is well-document and researched. Framing the problem in terms of linear model 
matrix notation allows researchers to benefit from this theory. When placed in the context of linear models, 
transformations can be used to manipulate model coefficients. The distributional properties can be derived 
and confidence intervals and prediction intervals can be formed for predicting solutions. 4 Finally, since most 
models can be expressed as some form of linear model, other methods of calibration can be evaluated and 
compared to the solution proposed here. 

A well-known theory available is weighted least squares. Weighted least squares multiplies the design 
matrix and the response variable by the square root of the covariance matrix. The transformed matrices 
then satisfy the constraints of the traditional linear model. The transformed values are used to solve the 
regression analysis. 

$ = 1 x'irV (2) 

The classical least squares solution was fit to the data. This solution did not any weighting scheme and 
treated all errors equally. Figure 3 shows when pressure is 0.12 psia, the percentage reading error is =b 12 
percent. 


Minimizing Reading Error Method GLS 



Figure 3. Residuals from GLS Appraoch 


In an effort to adjust the fit, it was thought to weight the design matrix in accordance with the allowed 
error. For large values of pressure, an error of 0.10 gives a small percentage reading error. At 5.00 psia, 0.05 
error gives a 1 percent reading error. However, an error of 0.05 at 1.00 psia gives a 5 percent reading error. 
Hence the full scale error that is acceptable at 5.00 psia is not acceptable at 1.00 psia. To reflect this, a 
weighting matrix was created. The weighting matrix places more emphasis on the points at lower pressures. 

For the case of minimizing reading error, the weighted least squares solution must be minimized. The 
weighting matrix used is not the traditional inverse square root covariance matrix. In this case, the weighting 
matrix is designed to reduce the error divided by pressure for each case. The weighting matrix is defined as 
one over the pressure for all cases explored here. 
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' A 0 ••• 0 ' 

0 ±- ... 0 

Weight Matrix = . F2 (3) 

0 0 ... 4 - 

1 n 

It is important to note that this weighting matrix is not unique. In this case, the weighting matrix directly 
expresses the matrix expression shown above. However, other forms of the weighting matrix can be chosen 
to tailor the model fit. For example, instead of weighting each value by the inverse pressure, it is possible 
to weight it by inverse pressure squared. The off-diagonal elements can also be adjusted to account for 
covariance structure of the observations. 


A. Alternative Approaches to Generalized Least Squares 

The following two approaches were investigated as alternatives to the GLS approach. 

• Weighted Response 

• Weighted Least Squares 

The first method attempted was to apply a weighting scheme to the response variable to minimize percentage 
reading. Upon first consideration, this approach seems like it would produce the smallest error terms. The 
procedure attempts to directly model the term error divided by pressure. Using the derivations below, it 
can be shown that this is equivalent to fitting the weighted response variable. Hence, dividing each response 
variable by pressure and fitting a function of the pressure and temperature values to this transformation is 
a direct method to minimize the error. 


Reading Error^ = 


Yi -Vi 

Pi 


yi - PjXjb 

Pi 



( 4 ) 


This solution method adds a small bias to the full scale error to decrease the prediction variation. Using 
this method assumes the design space is multiplied by the observed pressure matrix equation 3 to predict 
voltage. This model can be expressed in matrix theory as Equation 5 . 


(wy - Xb)' (toy - Xb) (5) 

This solution weights the response variable to minimize the reading error. Under this problem structure the 
solution is 


b = (X , X) -1 X , «;y (6) 

This solution is different from the traditional least squares solutions in that only the response variable is 
weighted and the weighting scheme is not a function of the covariance matrix. In Figure 4, the residuals 
from the weighted response approach are shown. The bias inherent in the solution is very apparent as the 
variation in the estimate is small but the bias can be plainly seen. At 0.12, 2.50, and 4.00 psia, the bias 
tends to be negative. The pressure values of 0.50, 0.79, and 1.00 psia tend to have a high bias. The bias 
introduced by weighting the response was too great to overcome. Hence, this result was discarded and a new 
method to minimize the percentage reading was proposed. 

An alternative approach is phrase the problem in terms of voltage error divided by pressure. This is 
proportional to pressure error when the value is divided by 2. The reading error for this case is expressed as 

Reading Error* = — (7) 

i M 

The matrix notation is given as 


(wy — wXjSy(wy — wX/3) (8) 

The estimate of the coefficients under this parameterization are 
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Minimizing Reading Error Weighted Response 



Figure 4. Residuals from Weighted Response Approach 


^=(XVX)“ 1 XVy (9) 

Notice that both the design matrix and the response variable are weighted. The weighting matrix takes 
the form of tradition least squares. The major difference here is the weighting matrix is not necessarily the 
covariance matrix. The weighting matrix equation 3 is scaled to weight the values of the response that have 
the lowest pressure. The idea is to weight the region of the design space that must have the lowest full-scale 
error in order to satisfy the reading error requirement. 

Figure 5 above shows very little improvement over the generalized least squares solution. Minimization 
that focused on improving the fit at the low pressure values did not significantly improve the result. Analysis 
of the residuals shows why. The best weighting methods can do is make the regression line pass through 
the center of the different points. A line passing through the center of the points will reduce the error to its 
lowest value. It appears that the GLS solution is already close to this center point. The weighting above 
moved the line slightly closer to the mean of the points and hence did not show much improvement. 

V. In-Flight Zero 

Analysis of the residuals showed the temperature changes were responsible for much of the error. The 
different values of temperature resulted in a different random draws from the normal distribution. This is 
expressed in the equations by the whole plot error term, 6 . The whole plot error term moves the resulting 
estimate up or down for each whole plot. To remove this, each set of temperature effects were treated as an 
indicator variable that increased or decreased the estimate average. Whole plot error caused by changing 
temperatures were removed using indicator variables. 

It is very easy to find the intercept adjustment for each observed whole plot when we have the tempera- 
tures, pressure, and voltage recorded. It will be another matter to find the adjustment in flight when only 
temperature and voltage are measurable. The answer to this is the in-flight zero value. Prior to atmospheric 
interface, measurements will be taken where pressure and temperature are known. The only instance where 
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Minimizing Reading Error Weighted Least Squares 



Figure 5. Residuals from WLS Approach 


pressure is known is in space where there is a hard vacuum. Since pressure and temperature will be known, 
these values can be used to derive the intercept adjustment, which will eliminate a significant amount of the 
variability in the calibration model. 

The model from Equation 1 above was reduced to account for the temperature effects being treated 
as indicator variables. Attempting to leave all temperature values in the model resulted in high variance 
inflation factors. As a result, the temperature model terms that are independent of pressure were removed 
from the model. The four terms were replaced by a set of indicator variables representing the temperature 
values. Interaction model terms involving temperature and pressure were allowed to remain in the model. 

V = 0Q + 01 P + flllP 2 + 012 P X Tsensor + 0i3 P x Tsse + € (10) 

Analysis of the data using the in-flight zero technique showed significant improvement. The graph below 
shows the reduction in variance by accounting for in-flight zero. The indicator variables account for the whole 
plot error leaving only the subplot error which is considerably smaller of the two. As a result, most of the 
variance associated with changes in temperature values can be removed. The resulting model still accounts 
for the location in the temperature design space by keeping the temperature by pressure interactions. These 
interactions show how changes in pressure and voltage are related to the location within the temperature 
design space. 

Figures 6 and 7 show the improvement by accounting for in-flight zero. The graphs show the new reading 
error to be =b 3 percent. Once the variance due to temperature is removed, the only variance remaining is 
due to subplot error. At this point, the weighting matrix, w, defined earlier can be used to force the linear 
estimate through the middle of the observed value. Since the variance of the points is much smaller, the 
improvement gained by the weighting is more noticeable and reduces the reading error down to d= 2 percent. 
Figure 8 and 9 shows the improvement of using inflight zero with WLS. 
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Minimizing Reading Error Method GLS In Flight Zero 



Figure 6. GLS with In-Flight Zero Correction 


Minimizing Reading Error Method GLS In Flight Zero 



Figure 7. GLS with In-Flight Zero Correction 
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Minimizing Reading Error WLS In Flight Zero 



Figure 8. WLS with In-Flight Zero Correction 


Minimizing Reading Error WLS In Flight Zero 



Figure 9. WLS with In-Flight Zero Correction 
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VI. Prediction Intervals 


Aside from reducing the variance, linear model theory allows the analyst to estimate how close the 
predicted value will be to the actual value. The methodology above allows the construction of prediction 
intervals around each estimate. Prediction intervals give an estimate of the difference between the estimated 
value and the next observed value. The prediction comes with a confidence level such that the analyst is 
95% sure the next observation will fall between the calculated bounds. 

When the predictor and response values are both known, a prediction interval is of little use. However 
when only the predictor values are know and the model must be used to estimate the response values, 
predictions intervals allow the analyst to judge the quality of the prediction. When two prediction intervals 
have the same level of confidence, the smaller interval is a more precise estimate. Hence prediction intervals 
give researchers a method of comparing estimators. 

Linear models provides the frame work to evaluate estimators by using prediction intervals. The model 
forms shown above and the corresponding variance weighting matrices can be evaluated in the prediction 
interval equation. Researchers can now evaluate the prediction interval widths of two estimators. Other 
estimators can be structured to align with linear model theory and their resulting prediction intervals can 
be compared. Hence this method gives the researcher the ability to determine which method should be used 
to predict future values. Prediction interval widths are defined as 


2t df aJl + X i A£A<X' i 

Interval Width* = v (11) 

■M 

The weighting matrix, w , can be combined with the design matrix to produce several options for the A 
matrix in the confidence interval width. It is not enough to reduce the confidence interval width. Equation 
6 showed that changing the application of the weighting matrix adding bias in an effort to reduce variation 
will not always produce an acceptable result. Hence the point estimate bias and prediction interval width 
should be taken into account jointly when evaluating the strength of a prediction interval. 

Bias, = (12) 

VII. Conclusion 

This paper demonstrates how to use linear model theory to estimate a calibration equation for a pres- 
sure transducer. The linear model approach is robust enough to handle multiple estimation procedures. 
Specifically, it was shown how the theory could adjust and weight the design space to strategically reduce 
errors in specific regions. In the case study, interest in improving low pressure measurements was dictate 
by the project’s reading error requirement. The analysis showed how the error terms were dominated by 
temperature effects. The linear model framework and the in-flight zero method allowed us to compensate 
for error terms dominated by temperature. Other forms of the weighting matrix could be used to target any 
area of the design space. 

Additionally, linear model theory can be used to predict future responses. Estimators created with the 
theory can be given prediction intervals that show where future errors are thought to be and the magnitude 
of those errors. The predictions have a confidence interval associated with them to help evaluate the overall 
quality of the prediction. While this project used linear model theory for the MEADS system, statistical 
linear models can be use in many applications and calibration problems. 
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